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Abstract 


This report focuses on the extent to which students who are academically far off track in 
fourth or eighth grade in Arkansas catch up by eighth or eleventh and twelfth grades. We studied 
two recent cohorts of Arkansas students whose eighth-grade ACT Explore® scores were more 
than one standard deviation below the ACT Explore Benchmark scores associated with being on 
track. We found that 9% or fewer of the students who were far off track in eighth grade attained 
the ACT College Readiness Benchmarks® by eleventh or twelfth grade. We did a similar 
analysis for two cohorts of students beginning in fourth grade, using scores on the Arkansas 
Benchmark Exams in literacy and mathematics in grade four and ACT Explore scores in reading 
and mathematics in grade eight. In this analysis, the percentage of far-off-track fourth-grade 
students who attained the ACT Explore Benchmark scores in grade eight ranged from 4% in 
mathematics to 6% in reading. We also found that students from at-risk groups — those who are 
low-income, African American, Hispanic, English language learners, or in special education — 
had lower catching up rates than their more advantaged peers. These results are of special 
concern because a large percentage of students from those groups are far off track in fourth and 
eighth grade. 

These findings should underscore the importance of policies and practices that focus on 
getting students off to a good start in the early grades. These practices are especially important 
for disadvantaged students. Ongoing research should identify practices that help to accomplish 
this goal, and state and local policy should support efforts to disseminate and implement those 
practices. 
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Catching Up to College and Career Readiness in Arkansas 1 
Introduction 

In recent years educators and policymakers have set a goal that students graduate from 
high school ready for college and careers. However, as a nation we fall short of achieving this 
goal, particularly for disadvantaged students. In the states where the highest percentages of 
students took the ACT Assessment R in 2012, for example, 45% of students in the two lowest 
family income categories met ACT’s College Readiness Benchmarks in English, 21% in 
mathematics, 24% in reading, and 18% in science. In Arkansas, where 8,566 students from one 
of those two family income categories took the ACT, 51% of those students in English, 23% in 
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mathematics, 28% in reading, and 19% in science met the ACT Benchmarks. 

A substantial body of research supports the idea that the path to college and career 
readiness begins in early childhood. Gaps in vocabulary development begin in very early 
childhood (Hart & Risley, 1995), and students entering kindergarten from disadvantaged 
backgrounds tend to lag behind their more advantaged peers in vocabulary and overall oral 
language development (Farkas & Beron, 2004; Dunham, Farkas, Hammer, Tomblin, & Catts, 
2007) as well as in early reading and mathematics skills and background knowledge (West, 
Denton, & Germino-Hausken, 2000). In turn, early reading and mathematics skills and 
background knowledge predict student success in the later grades (Duncan, Claessens, Huston, 
Pagani, Engel, Sexton, Dowsett, Magnuson, Klebanov, Feinstein, Brooks-Gunn, Duckworth, & 
Japel, 2007; Claessens & Engel, 2013; Grissmer, Grimm, Aiyer, Murrah, & Steele, 2010; Geary, 

1 This study uses data maintained by the Arkansas Department of Education and is published with its permission. 

2 These two income categories together consist of students with a self-reported family income of less than $36,000 a 
year. 

3 These statistics are based on the updated Benchmarks of 22 in reading and 23 in science calculated in Allen (2013). 
The states with the highest percentages of students taking the ACT were Colorado, Illinois, Kentucky, Michigan, 
North Dakota, Tennessee, and Wyoming. The data file contained the most recent ACT scores of students who were 
twelfth graders in 2012; ACT scores for students who did not take the ACT in twelfth grade came from earlier 
grades and years. 
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2011). With these findings in mind, recent ACT reports have focused on the importance of 
getting students off to a good start in preschool and the early elementary grades (Sawyer, 2008; 
Sawyer & Gibson, 2012; ACT, 2012a; Dougherty, 2013). 

Learning gaps that emerge early are likely to widen over time because of “Matthew 
effects,” whereby those who start out ahead are at a relative advantage in acquiring new 
knowledge (Stanovich, 1986). These effects can occur because students who already know about 
a topic often find it easier to leam new information on the same topic (Willingham, 2006), and 
because prior exposure to knowledge can motivate students to learn more (Durik & Matarazzo, 
2009; Maltese & Tai, 2010). In addition, in order to catch up, students who are academically off 
track must grow faster than students ahead of them. The lagging students must do double duty, 
catching up on content that they missed earlier while mastering newly taught curriculum. 
Students who are already on track do not carry this extra burden. 

This report follows up on the analysis in a recent ACT research report (Dougherty & 
Fleming, 2012). We used the recently updated ACT College Readiness Benchmarks (Allen, 
2013) as a measure of high school students’ academic preparation for two- and four-year 
colleges and other postsecondary training programs leading to skilled careers (ACT, 2006). 
These Benchmarks identify the ACT scores associated with a 50% probability of earning a B or a 
75% chance of earning a C in entry-level college courses corresponding to the ACT subject 
tested (Allen & Sconing, 2005). We used the corresponding College Readiness Benchmarks on 
ACT Explore as indicators of whether eighth grade students are on track to meet the ACT 
Benchmarks. The report focuses on students who start out far off track — scoring more than one 
standard deviation below the ACT Explore Benchmark in eighth grade in a given subject, or 
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more than one standard deviation below the fourth-grade state test score associated with a 50% 
probability of reaching the ACT Explore Benchmark in eighth grade. 

The next section discusses the methodology of our study. The two following sections 
look at results for students catching up in Grades 8-11/12 and Grades 4-8. Subsections of the 
report examine the percentage of far-off-track students from different demographic groups 
reaching the Benchmarks four years later and how far short of the Benchmarks the other students 
fell. Finally, the conclusion discusses implications of our findings for how educators and 
policymakers should think about intervention and accountability requirements. 

Methodology 

Students in the Analysis 

Grades 8-12, For the analysis of students catching up in high school, we used data from 
four cohorts of Arkansas students who took the ACT Explore test in Grade 8 in the 2006-07 or 
2007-08 school years and the ACT in Grade 11 or 12 (Table l). 4 This analysis was possible in 
Arkansas because state education officials provided student-level enrollment, state test, ACT 
Explore, and ACT data from the state’s longitudinal data system that could be matched across 
datasets and years using the state student ID. This made it possible to link the ACT Explore and 
ACT results to state enrollment and academic achievement test data, and to disaggregate ACT 
Explore and ACT results based on state-provided student demographic information. 


4 Students were followed through twelfth grade and their most recent ACT scores were used, so students listed as 
taking the ACT in eleventh grade did not take the ACT again in grade 12. 
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Table 1 

Arkansas Grades 8-11 and 8-12 Student Cohorts 


Student cohort 

School year 
taking ACT 
Explore 

School year 
taking 
the ACT 

Grade last 
taking 
the ACT 

Number of 
students* 

2007-2010 

2006-07 

2009-10 

11 

1,152 

2007-2011 

2006-07 

2010-11 

12 

9,029 

2008-2011 

2007-08 

2010-11 

11 

1,745 

2008-2012 

2007-08 

2011-12 

12 

9,121 

Total (4 cohorts) 



21,047 


* 0.3% of tested students with incomplete demographic records were dropped from the 
analysis. 


About 29% of enrolled Arkansas eighth graders were in the longitudinal cohorts in the 
study. 5 As might be expected, the students in the longitudinal cohorts in the study — representing 
students who stayed in school, followed a normal grade progression, remained in the state, and 
took both ACT Explore and the ACT — had lower percentages of students from typically at-risk 
groups than did the general population of eighth grade students. For example, students in the 
study were less likely to be low-income, African American, Hispanic, English language learners, 
or in special education (Table 2). 6 This means that any difficulty that students in the study had 
catching up would likely be amplified in the general student population, thus lending greater 
weight to the findings of this study about the difficulties experienced by far-off-track students. 


5 From Table 2, the 21,047 Arkansas cohort students were about 29% of the 71,674 total eighth grade population 
from the cohorts’ eighth grade years. 

6 In Table 2, the demographic characteristics of each student was taken from the eighth grade data, so that 
comparison between longitudinal and eighth grade snapshot cohorts use the same information on each student. 
Moving from left to right, students in each column of Table 2 are a subset of those in the previous column. 
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Table 2 

Demographics of Arkansas 8 th Grade Students from 2006-07 and 2007-08 School Years 



Total number of 
8th graders* 

8th graders taking 
ACT Explore 

8th grade ACT 
Explore takers in 
longitudinal 
cohorts 

Number of students* 

71,674 

39,352 

21,047 

Percent low-income 

52 

51 

42 

Percent African American 

23 

24 

22 

Percent Hispanic 

7 

7 

5 

Percent English language 
learners 

4 

3 

2 

Percent special education 

13 

10 

4 


* 0.5% of enrolled students with incomplete demographic records were dropped from the analysis. 


Grades 4-8. For the analysis of students catching up in upper elementary and middle 
school, we used data from two cohorts of Arkansas students who took the Arkansas Benchmark 
Exams (ABE) in literacy and mathematics in Grade 4 in the 2006-07 or 2007-08 school years 
and ACT Explore in Grade 8 four years later (Table 3). The population of students in the 
longitudinal cohorts who followed a normal grade progression and took the ACT Explore test in 
eighth grade was less at risk than the general population of all fourth grade tested students (Table 
4). However, this difference was not as great as in the high school cohorts shown in Table 2. 
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Table 3 

Arkansas Grades 4-8 Student Cohorts 


School year 
taking 4th 

Student cohort grade test 

School year 
taking ACT 
Explore 

Grade taking 
ACT Explore 

Number of 
students* 

2007-2011 2006-07 

2010-11 

8 

26,653 

2008-2012 2007-08 

2011-12 

8 

27,128 

Total (2 cohorts) 



53,781 

* Fewer than 0.1% of tested students with incomplete demographic records were 
the analysis. 

dropped from 

Table 4 




Demographics of Arkansas 4 th Grade Students from 2006-07 and 2007-08 School Years 


Total number of 
4th graders* 

Tested 4th 
graders 

Tested 4th 
graders in 
longitudinal 
cohorts 

Number of students 

72,973 

69,818 

53,781 

Percent low-income 

59 

59 

56 

Percent African American 

22 

22 

21 

Percent Hispanic 

9 

8 

8 

Percent English language 
learners 

6 

6 

6 

Percent special education 

17 

15 

11 


* Fewer than 0.1% of enrolled students with incomplete demographic records were dropped from 
the analysis. 


Disaggregation of Students into Demographic Groups 

Grades 8-12 and 4-8. In addition to looking at all tested students, we disaggregated the 
students in the four longitudinal cohorts and the ACT Explore tested population as a whole into 
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19 additional subgroup categories, making 20 categories altogether. The first set of eight 
categories consists of: 

1 . All students 

2. Low-income students 

3. Non-low-income students 

4. African American students 

5. Hispanic students 

6. Other students 7 

7. English language learners 

o 

8. Special education students 

An additional set of 12 categories disaggregates students by income, ethnicity, and gender: 

9. Low-income African American males 

10. Low-income African American females 

1 1 . Low-income Hispanic males 

12. Low-income Hispanic females 

13. Other low-income males 

14. Other low-income females 

15. Non-low-income African American males 

16. Non-low-income African American females 

17. Non-low-income Hispanic males 

7 The “Other” category consists of individuals who are neither African American nor Hispanic. In Arkansas, the 
great majority of those students are White. The three ethnic categories (4-6) are mutually exclusive, as are the two 
income categories (2 and 3). 

s In addition, we looked at results for the following additional four categories not reported on in this paper: Hispanic 
English language learners, non-Hispanic English language learners, non-English language learners, and students not 
in special education. 
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18. Non-low-income Hispanic females 

19. Other non-low-income males 

20. Other non-low-income females 

To keep the number of statistics in the paper to a manageable size, this report focuses on the first 
eight categories. However, information on students disaggregated by income, ethnicity, and 
gender is available in Appendix B. 

Division of Students into Academic Preparation Groups 

Grades 8-12. We classified eighth grade students into three academic preparation groups 
in each of four subject areas (English, mathematics, reading, and science) based on their 
performance on ACT Explore in these areas: 

• “On-Track” students met the College Readiness Benchmark score on ACT Explore 
(Table 5) in the subject. 9 

• “Off-Track” students missed the Benchmark by one standard deviation or less. 

• “Far-Off-Track” students scored more than a full standard deviation below the 
Benchmark. 10 


Q Readers should note that reaching the ACT College Readiness Benchmark on the eighth grade ACT Explore test 
does not imply that the student is college-ready in eighth grade, only that he or she is on track to being college ready 
on the ACT by eleventh or twelfth grade. 

10 Standard deviations were chosen as the yardstick because they provide a common metric across different grades 
and tests. A one-standard deviation difference in scores is quite large: in reading and mathematics, it is roughly the 
difference between scoring at the Basic and Proficient levels on the National Assessment of Educational Progress 
(NAEP), or between the 16 th and the 50 th percentiles on a norm-referenced standardized test. The size of a standard 
deviation on Explore was 4.2 points in English, 3.5 in mathematics, 3.9 in reading, and 3.3 in science. Standard 
deviations on the ACT were 6.4 points in English, 5.3 in mathematics, 6.2 in reading, and 5.1 in science. 
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Table 5 

Student Academic Preparation Levels on ACT Explore (Grade 8) 


Academic 
preparation level 

English 

Mathematics 

Reading 

Science 

On Track 

Met or exceeded the 
College Readiness 
Benchmark 

13 or above 

17 or above 

16 or above 

1 8 or above 

Off Track 

No more than one 
standard deviation 
below the 
Benchmark 

9- 12 

14- 16 

13 - 15 

15 - 17 

Far Off Track 

More than one 
standard deviation 
below the 
Benchmark 

8 or below 

13 or below 

12 or below 

14 or below 


For example, a score of 16 or better in ACT Explore Reading indicated that a student was 
On Track; Off Track students scored from 13 to 15, while students scoring 12 or below were 
classified as Far Off Track. Similarly, Table 6 shows the ACT scores indicating that a student is 
On Track (meeting the ACT College Readiness Benchmark), Off Track, or Far Off Track. These 
Benchmarks, first set in 2005, were updated in 2013 based on more recent data linking students’ 
ACT scores to their grades in credit-bearing first-year college courses (Allen, 2013; Allen & 
Sconing, 2005). In the update, the English and mathematics Benchmarks remained the same, 
while the ACT Benchmark in reading changed from 21 to 22 and in science from 24 to 23. 
Similarly, the eighth grade ACT Explore Benchmark changed from 15 to 16 in reading and 20 to 
18 in science. We used these updated Benchmarks for all years of data in this report. 
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Table 6 

Student Academic Preparation Levels on the ACT 


Academic 
preparation level 

English 

Mathematics 

Reading 

Science 

On Track 

Met or exceeded the 
College Readiness 
Benchmark 

1 8 or above 

22 or above 

22 or above 

23 or above 

Off Track 

No more than one 
standard deviation 
below the 
Benchmark 

12 - 17 

17-21 

16-21 

18-22 

Far Off Track 

More than one 
standard deviation 
below the 
Benchmark 

1 1 or below 

16 or below 

15 or below 

17 or below 


In ACT Explore English, few students scored at the Far Off Track level of 8 or below. 
Thus, we focused our analysis of Far Off Track students in the other three subjects. 

Grades 4-8. Similarly, we classified fourth grade students in the two Arkansas cohorts as 
On Track, Off Track, and Far Off Track based on their scores on the Arkansas Benchmark 
Exams (ABE). This classification was based on a direct link between students’ fourth grade ABE 
scores in literacy and mathematics and their eighth grade ACT Explore reading and mathematics 
scores. This was done by using logistic regression to identify the fourth grade ABE score in each 
subject associated with a 50% or better probability of meeting or exceeding the eighth grade 
Benchmark on ACT Explore in the corresponding subject. 11 Data from both cohorts was 


11 Students were classified into two categories based on whether they did or did not meet the ACT Explore 
benchmark in a subject, and a logistic regression model such as the one described in Allen (2013) was used to assess 
the probability of meeting the ACT Explore benchmark in the subject as a function of the student’s fourth grade 
ABE score in the same subject. The fourth grade ABE literacy test, which covers both reading and writing, was 
treated as the closest same-subject match to the eighth grade ACT Explore reading test. 
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combined for this analysis. This led to the identification of the fourth grade academic preparation 

12 

levels shown in Table 7. 

Table 7 

Student Academic Preparation Levels on the Grade 4 Arkansas State Test 


Academic 
preparation level 

Literacy 

Mathematics 

On Track 

Met or exceeded the 

772 

675 

College and Career 
Readiness Target 

or above 

or above 

Off Track 

No more than one standard 
deviation below the Target 

586 -771 

575 - 674 

Far Off Track 

More than one standard 
deviation below the Target 

585 

or below 

574 

or below 


Classification of Far-Off-Track Students Based on Their Amount of Catching Up 

Grades 8-12. For each Far Off Track student in a given subject, we calculated the 
difference between the student’s eighth grade ACT Explore score and the ACT Explore College 
Readiness Benchmark in that subject. For example, consider a hypothetical student with a score 
of 10 on the ACT Explore reading test. This student has a scale score gap of -6 relative to the 


12 These preparation levels will be slightly different from those identified using different years’ data or a different 
linking methodology, e.g., the one described in ACT (2012b) and Dougherty & Fleming (2012). Most of the 
difference between the On Track targets shown in Table 7 and those developed by Dougherty & Fleming (2012) is 
due to the updating of the Explore reading Benchmark from 15 to 16 and the use of fourth grade data from the 2007- 
OS school year as well as from 2006-07. When 2006-07 data are used with the new Benchmarks, and only the 
linking methodology is different, the On Track targets differ by 4 ABE scale score points in mathematics (about .04 
standard deviation) and 13 ABE score points in literacy (about .07 standard deviation). Incorporating data from the 
2007-08 school year in addition to the different linking methodology causes the On Track targets to differ by 16 
points in mathematics (about .16 standard deviation) and 15 points in literacy (about .08 standard deviation). 
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ACT Explore Benchmark score of 16. (The gap is negative to emphasize that the student falls 
short of the Benchmark.) If the same student scores 16 on the ACT, the student’s scale score gap 
is -6 relative to the ACT Reading Benchmark of 22, the same as the student’s scale score gap on 
ACT Explore. An ACT reading score of 19, on the other hand, would constitute a gap of -3, and 
the student would have closed half of the ACT Explore score gap. A student scoring at or above 
the Benchmark on the ACT would have closed all of the gap. 

Score gaps can also be measured in standard deviation units. In that case, we refer to 
them as z-score gaps. The definitions of Off Track and Far Off Track students in Tables 5-7 are 
based on z-score gaps. For example, our hypothetical student with an ACT Explore reading score 
of 10 has a z-score gap of about -1.54 (i.e., -6/3.9, where 3.9 is the standard deviation of ACT 
Explore reading scores). This places the student in the Far Off Track group, as the student scores 
more than one standard deviation below the Benchmark. If the same student scores 16 on the 
ACT, the scale score gap is unchanged but the z-score gap narrows to -.97 (i.e., -6/6.2, where 
6.2 is the standard deviation of ACT reading scores), and the student is counted in the Off Track 
group. Z-score gaps adjust for the wider dispersion of student scores on the ACT than on ACT 
Explore. Analogously, a 15-pound weight gap is a larger share of typical weight differences 
among 5 -year-olds than it is among 30-year-olds, so a five-year-old who is 15 pounds 
overweight might be considered to be “more overweight” than a 30-year-old who is 15 pounds 
overweight. This may be small consolation to the 30-year-old who must still make the effort to 
lose 15 pounds. So both score gaps and z-score gaps are useful measures of how far students fall 
short of On Track Benchmarks. 

13 “Scale scores” are the familiar ACT Explore and ACT scores reported on a scale from 1 to 25 and 1 to 36, 
respectively. They are distinguished from “raw scores” which represent the percentage of test items correct, and 
other forms of score reporting such as stanines, percentile ranks, and grade equivalents that are used on norm- 
referenced tests. 
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Using score gaps as the metric, we classified students who scored Far Off Track on a 
given academic subject on ACT Explore into four scale score growth categories based on how 
much they closed their scale score gaps in the same subject when they took the ACT (Table 8). 
Likewise, we divided those students into four z-score growth categories based on how close they 
came to reaching the ACT Benchmarks (Table 9). 

Table 8 


ACT Explore-ACT Scale Score Growth Categories for Far Off Track Students 


Category 1 

“Reached Benchmark”: the student closed the entire ACT Explore 
scale score gap by scoring at or above the ACT College Readiness 
Benchmark. 

Category 2 

“Closed half or more of gap”: the student’s ACT Explore scale 
score gap narrowed by half or more on the ACT. 14 

Category 3 

“Closed gap by less than half’: the student’s ACT Explore scale 
score gap narrowed on the ACT, but by less than half. 

Category 4 

“No gap closing”: the student’s ACT Explore scale score gap stayed 
the same or widened on the ACT. 


14 This is a less stringent criterion than proposed in ACT (2009), which suggested that Off-Track students be 
expected to close half the point gap between Explore and Plan and half of the gap again between Plan and the ACT, 
or three-quarters of the gap altogether between Explore and the ACT. 



14 


Table 9 

ACT Explore-ACT z-Score Growth Categories for Far Off Track Students Based on the Change 
in the Student ’s Academic Performance Level 


Category 1 

“Reached Benchmark”: the student moved from Far Off Track on 
ACT Explore to scoring at or above the ACT College Readiness 
Benchmark. 

Category 2 

“Reached top half of Off Track level”: the student moved from Far 
Off Track to scoring in the top half of the Off Track performance 
level on the ACT. 15 

Category 3 

“Reached bottom half of Off Track level”: the student moved from 
Far Off Track to scoring in the bottom half of the Off Track 
perfonnance level on the ACT. 

Category 4 

“Stayed Far Off Track”: the student remained in the Far Off Track 
category on the ACT. 


To see how students are placed in these categories, consider our far-off-track eighth- 
grade student with an ACT Explore reading score of 10, representing a scale score gap of -6 and 
a z-score gap of -1.54 relative to the Explore reading benchmark of 16. If the student later scores 
at or above the College Readiness Benchmark of 22 on the ACT reading test, that student attains 
Category 1 on both growth metrics. To reach Category 2 on scale score growth, the student must 
score 19-21 on the ACT, reducing the scale score gap relative to the Benchmark to 3 points or 
less. To reach Category 2 on z-score growth, the student must score no more than one-half 
standard deviation below 22, also a score of 19-21. (A half standard deviation in ACT reading is 
6.2/2 = 3.1 points.) For Category 3 on scale score growth, the student must score 17 or 18 on the 
ACT; a score of 16 or below would fail to narrow the 6-point gap relative to the Benchmark, thus 
leaving the student in Category 4. In comparison, an ACT score of 16-18 places the student in 


15 The ACT scores required to reach this category are 15 in English, 20 in mathematics, 19 in reading, and 21 in 
science. See the standard deviations listed in footnote 10. 
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Category 3 on z-score growth, since the borderline between Category 3 and 4 — one standard 
deviation below the Benchmark — falls between an ACT score of 15 and 16. 16 

Scale score growth measures between two tests depend on the tests having a common 
vertical scale, as is the case for ACT Explore and the ACT. Z-score growth measures can be used 
even if no such scale exists, as long as the subject matter of the two tests is similar enough for 
the concept of “growth” to be meaningful. 

Because the scores of any predefined group of students contains an error component that 
is positive on average for students chosen from near the top of the score distribution and negative 
on average for students chosen from near the bottom, all averages of groups of students chosen 
based on their prior performance tend to move back toward the average of all students. Using a 
sports analogy, a group of baseball players chosen for the highest batting averages in the first six 
weeks of the season will probably bat at a lower average for the rest of the season, even if they 
continue to bat well above the average for all players (Campbell & Kenny, 1999). This 
regression effect tends to reduce the expected future growth of On Track students and increase 
the expected future growth of Far Off Track students. 17 Possibly offsetting this regression effect 
are Matthew effects, which give an advantage to the students with better prior academic 
preparation. 18 


16 Note that the z-score categories only depend on the student’s ending point, whereas the scale score categories also 
depend on how far behind the Far-Off-Track student starts. 

17 In eighth grade, the cut scores for being On Track (shown in Table 5) were slightly below the Arkansas Explore 
score mean of 13.6 in English but above the Arkansas means of 14.6 in mathematics, 13.8 in reading, and 15.7 in 
science. Thus any On Track score was above the state average in three of the four subjects. 

18 Faster growth of groups of previously higher performing students than of groups of lower performing students is 
highly suggestive of Matthew effects. In fact, because of regression to the mean, the same growth by groups of 
previously higher performing students could be suggestive of these effects. On the other hand, the fact that 
individual scores diverge over time is not in itself proof of Matthew effects, as individual scores will spread out over 
time even if current period score growth is uncorrelated with growth or performance levels in prior periods - think 
of the spreading out of an ink blot even if all movement of ink molecules is random and unrelated to prior position 
or movement. Note also that growth comparisons of student groups that begin at different levels depend on the 
assumption that the score scale has equal-interval properties - that is, growth from 10 to 15 has the same meaning as 
growth from 20 to 25. 
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Scale scores but not z-scores can be used to look for Matthew effects, as z-scores “adjust 
out” the increased spread of scores over time, including any divergence of groups of students that 
might be attributable to Matthew effects. Z-score measures do not, on the other hand, remove 
regression effects. Therefore any z-score measure will always show the top students declining 
and the bottom students improving, even if their scale scores diverge. Thus, it is not sufficient 
just to see if Far Off Track students’ z-scores improve, but whether they improve enough to 
bring them close to the Benchmarks. That is the reason for emphasizing the student’s ending 
point when looking at z-scores. 

The academic preparation level measures (Far Off Track, Off Track, and On Track) and 
growth category measures used in this report are subject-specific. A student might be Far Off 
Track in ACT Explore reading and/or achieve little growth in that subject between ACT Explore 
and the ACT, but perform very well in ACT Explore mathematics and/or achieve strong 
mathematics growth between ACT Explore and the ACT. 

Grades 4-8. We used students’ Arkansas Benchmark Exam scores in literacy and 
mathematics in Grade 4 and the ACT Explore scores in reading and mathematics in Grade 8 as 
our endpoints for student growth. As the ABE and ACT Explore tests are not scored on a 
common vertical scale, no scale score growth measure exists between the two sets of tests. 19 
However, academic performance levels and growth measure can be calculated based on the 
number of standard deviations that students scored below the On Track measure on each test. 
Thus, fourth grade Far Off Track students were classified into the four z-score growth categories 
shown in Table 9 based on the performance level they reached on ACT Explore in eighth grade. 

19 For example, while the Explore is scored on a scale from 1 to 25 in each subject, the 2008 fourth grade ABE scale 
scores ranged from 24 to 987 in literacy and 113 to 966 in mathematics. Also, the fourth grade ABE literacy and 
eighth grade ACT Explore reading tests cover overlapping but not identical subject matter, as the literacy exam 
covers reading and writing and the ACT Explore reading test covers reading only. 
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Because z-score growth measures were available at both levels (grades 4-8 and 8-12), we 
focus on those measures in the main body of this report. Appendix A contains a discussion of 
scale score growth by Arkansas students in grades 8-12. 

Results 

Closing Academic Preparation Gaps in High School 
What Percentage of Students Were Far Off Track in 8 th Grade? 

Table 10 shows the percentage of ACT Explore eighth grade test-takers in Arkansas who 
were Far Off Track in the 2006-07 and 2007-08 school years, the starting years for the students 
in the four cohorts in this study. Between 30% and 43% of students in the overall population 
were Far Off Track in 2006-07 and 2007-08 (top row of Table 10), depending on the subject 
tested. These percentages were considerably higher for students in at-risk student groups, as can 
be seen from the remaining rows. For example, roughly 41, 55, and 45 percent of low-income 
students were Far Off Track in mathematics, reading, and science, respectively. 
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Table 10 

Percentage of Arkansas ACT Explore Tested 8 th Grade Students Who Were Far Off Track from 
2006-07 and 2007-08 School Years 





Percentage of 8th graders who 
Far Off Track 

were 

Group 

Category 

Number of 
students 

Mathematics 

Reading Science 

1 

All students 

39,352 

30 

43 

35 

2 

Low-income 

19,941 

41 

55 

45 

3 

Non-low-income 

19,411 

20 

30 

25 

4 

African American 

9,261 

50 

66 

54 

5 

Hispanic 

2,612 

37 

52 

40 

6 

Other 20 

27,479 

23 

34 

28 

7 

English language learners 

1,179 

54 

70 

52 

8 

Special education 

3,900 

78 

83 

72 


As Table 1 1 shows, the percentages of Far Off Track students were lower for students in 
our longitudinal cohorts, who stayed in school, made normal progress through the grades, 
remained in the state, and took the ACT college readiness assessment in grade 11 or 12. Not only 
were the students in these cohorts less likely to be from at-risk groups (Table 2), but the 


20 As mentioned in an earlier footnote, the “Other” category consists of students who are neither African American 
nor Hispanic. Thus, the sum of the number of students in the African American, Hispanic, and Other categories 
equals the total number of students in the top row. Likewise, the number of low-income and non-low-income 
students adds up to the number in the top row. 
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longitudinal cohort students from at-risk groups were less likely than others in their demographic 

2 1 

groups to be Far Off Track." 

Table 1 1 

Percentage of Arkansas ACT Explore Tested 8 th Grade Students Who Were Far Off Track from 
Students in Longitudinal Cohorts, 2006-07 and 2007-08 School Years 


Group 

Category 

Number of 
students 

Percentage of 8th graders who were 
Far Off Track 

Mathematics Reading Science 

1 

All students 

21,047 

17 

30 

23 

2 

Low-income 

8,739 

26 

42 

32 

3 

Non-low-income 

12,308 

11 

21 

16 

4 

African American 

4,652 

36 

55 

42 

5 

Hispanic 

1,085 

24 

38 

27 

6 

Other 

15,310 

11 

22 

16 

7 

English language learners 

361 

36 

58 

39 

8 

Special education 

865 

64 

74 

58 


21 For example, the percentages of Far Off Track students among eighth grade low-income students not in the 
longitudinal cohorts were 46%, 58%, and 49% in mathematics, reading, and science, respectively. 
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What Percentage of Far Off Track 8 th Graders Reached College Readiness Benchmarks by 
11 th or 12 th Grade? 

Based on an analysis of the Arkansas longitudinal cohorts, few Far Off Track eighth 
grade students reached the ACT College Readiness Benchmarks by eleventh or twelfth grade. 
Only about 3% of Far Off Track eighth graders met the ACT Benchmark in mathematics, 9% in 
reading, and 6% in science (Figure 1). Success rates for Off Track students were higher: 19% 
in mathematics, 32% in reading, and 18% in science. By contrast, the majority of On Track 
eighth graders met the ACT Benchmarks in Grade 11 or 12. 


100 



On Track Off Track Far Off Track 

Preparation level in grade 8 
□ Mathematics H Reading □ Science 


Figure 1. Percent of Arkansas students meeting the ACT College Readiness Benchmarks in 
grades 11-12, given their eighth grade ACT Explore performance 


21 Results for English are not included in this chart because, as noted earlier, few students scored at the Far Off 
Track level of 8 or below on the Explore English exam. 
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Figure 1 combines results for students taking the ACT for the final time in twelfth grade 
(about 86% of students in the longitudinal cohorts) and those taking the ACT for the last time in 
grade 1 1 . The twelfth grade ACT takers might be expected to do better because they had more 
time to grow between ACT Explore and the ACT (an average of 4.1 vs. 3.2 years, respectively). 
On the other hand, the eleventh graders may have benefitted from selection bias, as they may 
have had many of the stronger students who took the ACT early and were satisfied with their 
scores. In fact, the twelfth grade ACT takers did better: rounding off to the nearest percent, 
twelfth grade ACT takers from the eighth grade On Track group stayed on track at a rate of 71%, 
76%, and 66% in mathematics, reading, and science, respectively, versus 63%, 74%, and 65% 
for their eleventh grade counterparts; Far Off Track students taking the ACT in twelfth grade 
reached the Benchmark at a rate of 3%, 9%, and 6% in those three subjects, versus 1%, 6%, and 
5% for students taking the ACT for the last time in eleventh grade. 23 

How Did the Percentage of Far Off Track 8 th Graders Reaching Benchmarks by 11 th or 
12 th Grade Vary Across Student Demographic Groups? 

Figure 2 shows how the percentages of Far Off Track eighth grade students reaching 
College Readiness Benchmarks by eleventh or twelfth grade varied between low-income and 
non-low-income students. Figure 3 provides the same information by student ethnic category, 
and Figure 4 provides this information for English language learner and special education 


23 Selection bias favoring students who took their last test in eleventh grade appears to be small. Average eighth 
grade Explore scores in mathematics, reading, and science for those students were 15.9, 15.1, and 16.8, respectively, 
versus 15.9, 14.8, and 16.6 for students taking the ACT for the last time in twelfth grade. Looking at scale score 
growth between Explore and the ACT, students taking the ACT for the last time in eleventh grade had more growth 
per year (1.2, 1.6, and 1.0 points in reading, mathematics, and science, versus 1.1, 1.5, and 0.9 for the twelfth grade 
group). However, because they had more time to grow, the twelfth grade group had higher overall score growth 
between Explore and the ACT (4.3, 6.0, and 3.7 points in reading, mathematics, and science, versus 3.8, 5.1, and 3.1 
points for the eleventh graders). 
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24 

students. These charts show that Far Off Track students from at-risk groups such as low- 
income students, minority students, English language learners, and special education students 
reached the Benchmarks at lower rates than did their less at-risk counterparts. This is a matter of 
concern given that students from these groups are more likely to be Far Off Track in the first 
place (Tables 10 and 1 1). 
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Low - income 
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Student population 

□ Mathematics H Reading □ Science 

Figure 2. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1 or 12, by student income category. 


~ 4 The large amount of white space in these charts is deliberate, intended to emphasize the low rate of catching up of 
Far Off Track students from all demographic categories. 
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African American Hispanic Other 

Student population 

□ Mathematics □ Reading □ Science 


Figure 3. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1 or 12, by student ethnic category. 



Figure 4. Percent of Far Off Track eighth grade students meeting College Readiness Benchmarks 
on the ACT in grade 1 1 or 12, for English language learners and special education students. 
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How Much Growth Towards College Readiness Benchmarks Did Far Off Track Students 
Achieve in High School? 

To address this question, we disaggregated students into the z-score growth categories 
described in Table 9. We examined how many Far Off Track students either reached or moved 
up close to the Benchmark, as represented by the top two z-score growth categories. These 
students are shown in the first and second bar segments in Figures 5-8 on pages 25 and 26 of this 
report. (The “Reached Benchmark” category in these charts shows the same statistics as in 
Figures 1-4.) For example, the overall percentage of Far Off Track students in the top two z- 
score growth categories (reaching the Benchmark or moving to no more than a half standard 
deviation below it) was 6% in mathematics, 25% in reading, and 15% in science (Figure 5). For 
low-income students, the corresponding totals were 4% in mathematics, 20% in reading, and 
11% in science (Figure 6). 

It is also useful to look at the percentage of Far Off Track students who remained Far Off 
Track, represented by the lowest growth category in Table 9 and the last bar segment in Figures 
5-8. For low-income students, these percentages were 77% in mathematics, 52% in reading, and 
62% in science (Figure 6). African American and special education students were the most at- 
risk groups based on the percentage of students staying Far Off Track in high school: 83, 58, and 
67 percent of African American students remained Far Off Track in mathematics, reading, and 
science, respectively (Figure 7), while the corresponding statistics for special education students 

25 

were 88, 69, and 73 percent (Figure 8). 


25 As discussed in the methodology section, score metrics based on z-scores cannot be used to examine Matthew 
effects, as dividing by the standard deviation removes the effect of increasing variance in test scores over time. 
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Figure 5. Percent of Arkansas Far Off Track students changing academic preparation levels in 
grades 8-11 or 8-12, by subject. 
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Figure 6. Percent of Arkansas Far Off Track students changing academic preparation levels in 
grades 8-11 or 8-12, by subject and income. 
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Figure 7. Percent of Arkansas Far Off Track students changing academic preparation levels in 
grades 8-11 or 8-12, by subject and ethnicity. 
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Figure 8. Percent of Arkansas Far Off Track English language learners and special education 
students changing academic preparation levels in grades 8-11 or 8-12, by subject. 
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Closing Academic Preparation Gaps between Grades 4 and 8 
What Percentage of Students Were Far Off Track in 4 th Grade? 

Table 12 shows the percentage of fourth grade ABE test- takers in Arkansas who were Far 
Off Track in the 2006-07 and 2007-08 school years, the starting years for the students in the two 
Grades 4-8 cohorts in this study. 36% and 44% of students were Far Off Track in mathematics 
and literacy, respectively (top row of Table 12). These percentages were higher for students in 
at-risk student groups, as can be seen from the remaining rows. Students in the longitudinal 
cohorts (Table 13) were Far Off Track at lower rates, reflecting the better initial preparation of 

this subset of students who followed a nonnal grade progression between grades 4 and 8 and 

26 

took ACT Explore in eighth grade . 


26 An exception was students in special education, for whom students in the longitudinal cohorts were Far Off Track 
at higher rates. Data on attrition of special education students by type of disability might be helpful in casting light 
on this issue. 
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Table 12 

Percentage of Arkansas Tested 4 th Grade Students Who Were Far Off Track from 2006-07 and 
2007-08 School Years 





Percentage of 4th graders 
who were Far Off Track 

Group 

Category 

Number of 
students 

Mathematics 

Literacy 

1 

All students 

69,534 

36 

44 

2 

Low-income 

40,798 

47 

55 

3 

Non-low-income 

28,736 

22 

28 

4 

African American 

15,424 

59 

64 

5 

Hispanic 

5,889 

46 

58 

6 

Other 

48,221 

28 

36 

7 

English language learners 

4,502 

53 

66 

8 

Special education 

10,736 

63 

75 
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Table 13 

Percentage of Arkansas Tested 4 th Grade Students Who Were Far Off Track from Students in 
Longitudinal Cohorts, 2006-07 and 2007-08 School Years 


Group 

Category 

Number of 
students 

Percentage of 4th graders 
who were Far Off Track 

Mathematics Literacy 

1 

All students 

53,781 

33 

40 

2 

Low-income 

30,201 

43 

51 

3 

Non-low-income 

23,580 

20 

26 

4 

African American 

11,103 

56 

61 

5 

Hispanic 

4,534 

43 

55 

6 

Other 

38,144 

25 

32 

7 

English language learners 

3,399 

50 

64 

8 

Special education 

5,835 

68 

81 


However, the differences in Far Off Track rates for the same student groups between 
Tables 12 and 13 are less than those between Tables 10 and 11, most likely because student 
attrition from the grades 4-8 cohorts was less than in high school. For example, the 53,781 
students in the “All Students” group in Table 13 represents 77% of the 69,534 students in Table 
12, whereas the 21,047 students in the corresponding group in Table 11 represent 53% of the 
39,352 students in Table 10. In addition, cohort attrition may be more closely related to academic 
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performance in high school (e.g., students drop out, are retained in grade, or choose not to take 
the ACT) than in the middle grades. 

What Percentage of Far Off Track 4 th Graders Were On Track by 8 th Grade? 

For the Arkansas longitudinal cohorts in the study, only about 4 and 6 percent of Far Off 
Track fourth graders reached the ACT Explore College Readiness Benchmarks by eighth grade 
in mathematics and reading, respectively (Figure 9). Success rates for Off Track students were 
higher: 30% in mathematics and 31% in reading. As was the case in high school, the majority of 
On Track fourth graders were still on track four years later. 
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Preparation level in grade 4 
□ Mathematics □ Literacy/Reading 


Figure 9. Percentage of Arkansas students meeting the ACT College Readiness Benchmarks on 
Grade 8 ACT Explore, given their fourth grade ABE performance. 
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How Did the Percentage of Far Off Track 4 th Graders Getting On Track by 8 th Grade Vary 
Across Student Demographic Groups? 

Figure 10 shows how the percentages of Far Off Track fourth grade students getting on 
track by the eighth grade ACT Explore varied between low-income and non-low-income 
students. Figure 11 provides the same information by student ethnic category, and Figure 12 
provides this information for English language learner and special education students. These 
charts show that Far Off Track students from at-risk groups caught up at lower rates than did 
their less at-risk counterparts. Students from these groups were also more likely to be Far Off 
Track in the first place, as shown in Tables 12 and 13. 27 
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Figure 10. Percentage of Far Off Track fourth grade students getting On Track by the 8 th grade 
ACT Explore, by student income category. 


27 Results for the other disaggregated student categories are available in Appendix B. 
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African American Hispanic Other 

Student population 

□ Mathematics 11 Literacy/Reading 


Figure 11. Percentage of Far Off Track fourth grade students getting On Track by the 8 th grade 
ACT Explore, by student ethnic category. 



Figure 12. Percentage of Far Off Track fourth grade students getting On Track by the 8 th grade 
ACT Explore, for English language learners and special education students. 
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How Much Growth Towards 8 th Grade Benchmarks Did Far Off Track Students Achieve 
between Grades 4 and 8? 

Using the growth categories in Table 9, we examined the percentage of Far Off Track 
students who either reached the Benchmark or moved up into the top half of the Off Track 
category, indicating that they were getting close to the Benchmark (Figures 13-16 on pages 34 
and 35). (The “reached Benchmark” category in these charts reports the same statistics as in 
Figures 9-12.) The overall percentage of Far Off Track students in the top two growth categories 
was 12% in mathematics and 1 1% in literacy/reading (Figure 13). For low-income students, the 
corresponding totals were 1 1% in mathematics and 9% in literacy/reading (Figure 14). 

It is also useful to look at the percentage of Far Off Track students who remained Far Off 
Track, represented by the last bar segment in Figures 13-16. For low-income students, these 
percentages were 53% in mathematics and 70% in reading/literacy (Figure 14). As was the case 
in high school, African American and special education students were the most at-risk groups 
based on the percentage of students staying Far Off Track in grades 4-8: 57 and 77 percent of 
African American students remained Far Off Track in mathematics and reading/literacy, 
respectively (Figure 15), while the corresponding statistics for special education students were 
71 and 78 percent (Figure 16). 


28 As noted earlier, rounding may cause totals in the charts to differ from 100% and subtotals to differ from those 
reported in the text. For example, in mathematics in Figure 13, 3.682% of students in the first category and 8.817% 
in the second category add up to 12.499% in the two categories combined. 
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Figure 13. Percent of Arkansas Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject. 
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Figure 14. Percent of Arkansas Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject and income. 
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Figure 15. Percent of Arkansas Far Off Track students changing academic preparation levels 
between Grades 4 and 8, by subject and ethnicity. 
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Figure 16. Percent of Arkansas Far Off Track English language learners and special education 
students changing academic preparation levels between Grades 4 and 8, by subject. 
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How Did Growth by Far Off Track Students in the Middle Grades Compare with Growth 
by Far Off Track Students in High School? 

Growth comparisons between grades 4-8 and 8-11/12 can be difficult to make because of 
differences in selection effects between the two levels. These selection effects ought to favor 
growth by students in high school cohorts, as attrition is greater in high school. 29 Attrition is 
likely to remove a disproportionate share of less prepared and slower growing students, who are 
more likely to drop out, be retained in grade, and not take the ACT. 

A second issue complicating comparisons between grades 4-8 and 8-11/12 is differences 
in the content alignment of the fourth grade ABE tests with eighth grade ACT Explore, 
compared with the alignment of ACT Explore with the ACT. Differences in content tested 
reduce the correlation between the results from earlier and later tests, increasing regression 
effects (Campbell & Kenny, 1999). Thus, lower alignment of the grades 4 and 8 tests would 
produce more “growth” between grades 4 and 8 for the students who are farthest off track, 
compared with the growth shown by similar students between grades 8 and 1 1/12 on more highly 
aligned tests. This effect is likely to work in the opposite direction from selection effects, 

30 

favoring growth in the middle grades over high school. 

With these caveats, Tables 14 and 15 (on pages 38 and 39) compare the percentage of 
students from each group who made it into the top two perfonnance levels by the end of the 
period (grade 8 for fourth graders or grades 11-12 for eighth graders), summarizing information 

29 The greater selectivity of the high school cohorts can be seen by comparing the size of the cohorts with total grade 
level enrollment in Tables 2 and 4. Also, in grade 8 the students in longitudinal cohorts did better relative to all 
tested students (Table 11 vs. Table 10) than was the case in grade 4 (Tables 13 vs. Table 12). These effects result in 
smaller groups of Far Off Track students in high school than in grades 4-8, as shown in Tables 14 and 15. 

30 Correlations between grade 4 ABE and grade 8 ACT Explore scores were slightly lower than between grade 8 
ACT Explore and grades 11/12 ACT scores: the former correlations were .681 and .650 in mathematics and 
reading/literacy, respectively, versus .692 and .691 between grades 8 and 11/12. 
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from Figures 5-8 and 13-16. In addition, these tables provide information on how far below the 
On Track level each group of students started out on average, measured in standard deviation 
units. This information is shown in the columns labeled “average distance below Benchmark in 
grade 4 (or 8).” 

In mathematics, the students in the grades 4-8 cohorts started out about the same distance 
behind as did students in the grades 8-11 or 8-12 cohorts, but in general were more successful at 
making it into the top two levels by the end of the cohort period. For example, low-income 
students in the grades 4-8 cohorts started out an average of 1.73 standard deviations below On 
Track perfonnance levels in fourth grade; their counterparts in the grades 8-11 and 8-12 cohorts 
started out about 1.77 standard deviations behind (Table 14). Yet 11% of low-income students 
were able to transition into the top two perfonnance levels in the middle grades, versus 4% in 
high school. This provides some evidence that it may be easier to catch students up in 
mathematics in the middle grades than in high school. 

In reading, students caught up at higher rates in high school, but also didn’t start out as 
far behind (Table 15). An ordinary least squares regression of z-score growth versus initial 
scores provides evidence that Arkansas Far Off Track students who were equally far behind in 

ii 

reading did more catching up in the middle grades than in high school. However, differences in 
content alignment are likely to be an important issue affecting this comparison, as the fourth 
grade literacy test covers both reading and writing, while the ACT Explore and ACT reading 
tests only cover reading. Thus, further evidence is needed to detennine whether catching students 


31 For example, a student who started out 1.5 standard deviations below the On Track level in fourth grade was 
predicted to move .57 standard deviations closer to the Benchmark between fourth and eighth grade, while a student 
in a similar position in eighth grade was predicted to move .47 standard deviations closer to the Benchmark between 
grade 8 and grade 11 or 12. Similar results obtained when score change per year was used as the metric to allow for 
the additional time to grow for students taking the ACT in grade 12. 
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up in reading is easier in the middle grades than in high school, and whether current efforts to 
remediate students in middle school are more effective than similar efforts in high school. 

Table 14 

Percentage of Arkansas Far Off Track Students Reaching the Top Two Mathematics 
Performance Levels: Grades 4-8 versus Grades 8-12 


Category 

Number 
of Far Off 
Track 
students 

Grades 4-8 
Average 
distance 
below 
benchmark 
in grade 4 

Percent in 
top two 
growth 
categories 

Grades 8-11 and 8-12 
Average 

Number distance Percent in 

of Far Off below top two 

Track Benchmark growth 

students in grade 8 categories 

All students 

17,626 

-1.68 

12% 

3,664 

-1.74 

6% 

Low-income 

12,921 

-1.73 

11% 

2,282 

-1.77 

4% 

Non-low-income 

4,705 

-1.56 

18% 

1,382 

-1.68 

9% 

African American 

6,176 

-1.81 

8% 

1,673 

-1.81 

2% 

Hispanic 

1,937 

-1.71 

15% 

256 

-1.64 

7% 

Other 

9,513 

-1.59 

15% 

1,735 

-1.68 

10% 

English language 
learners 

1,691 

-1.76 

15% 

131 

-1.71 

5% 

Special education 

3,947 

-2.02 

5% 

555 

-2.08 

3% 
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Table 15 

Percentage of Arkansas Far Off Track Studen ts Reaching the Top Two Reading Performance 
Levels: Grades 4-8 versus Grades 8-12 


Category 

Number 
of Far Off 
Track 
students 

Grades 4-8 
Average 
distance 
below 
benchmark 
in grade 4 

Percent in 
top two 
growth 
categories 

Grades 8-11 and 8-12 
Average 

Number of distance Percent in 
Far Off below top two 

Track Benchmark growth 

students in grade 8 categories 

All students 

21,559 

-1.73 

11% 

6,269 

-1.31 

25% 

Low-income 

15,430 

-1.78 

9% 

3,695 

-1.33 

20% 

Non-low-income 

6,129 

-1.61 

16% 

2,574 

-1.28 

32% 

African American 

6,732 

-1.82 

5% 

2,552 

-1.35 

16% 

Hispanic 

2,469 

-1.81 

11% 

412 

-1.31 

16% 

Other 

12,358 

-1.67 

14% 

3,305 

-1.27 

33% 

English language 
learners 

2,136 

-1.85 

11% 

210 

-1.34 

11% 

Special education 

4,731 

-2.21 

6% 

637 

-1.43 

14% 


Conclusion 

The results in this study on the difficulty of catching up Far Off Track students are 
consistent with previous research findings (Sawyer, 2008; ACT, 2008, 2012a; Dougherty, 2010; 
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Dougherty & Fleming, 2012; Sawyer & Gibson, 2012). This study extends those findings to 
demographic subgroups such as low-income students, African Americans, Hispanics, English 
language learners, and special education students. When more states provide the necessary data, 

32 

research on students catching up by demographic subgroups in those states will be possible. 

These results support a general finding that it is difficult for students who are far behind 
to get on track in middle or high school. While overestimating the difficulty of catching up might 
encourage educators and policymakers to give up on students, underestimating the difficulty 
might lead educators to choose strategies and interventions that are too little and too late. For 
their part, policymakers who think that catching students up is easier than it actually is may 
reduce funding for educational programs. They may also hold schools to accountability targets 
that are not attainable over the period in question, creating strong incentives for leaders at 
various levels in the system to seek to artificially inflate test scores. 

The high percentage of students who are below college and career readiness achievement 
targets at all grade levels — and the difficulty of catching them up — should also lead educators 
and policymakers to focus on early learning and to emphasize prevention over remediation 
(ACT, 2012a). These prevention strategies may include: changing the regular academic program 
to give every student access to a content- and vocabulary-rich curriculum beginning in the early 
years (Willingham, 2009; Common Core State Standards Initiative, 2010; ACT, 2012c; 
Dougherty, 2013); strengthening the early reading and mathematics program in preschool 
through third grade; and implementing programs and strategies that improve students’ attendance 
and academic behaviors (Diamond, Barnett, Thomas, & Munro, 2007a, 2007b; Durlak, 

32 For example, the Kentucky Department of Education provided similar data, making possible a research report on 
students catching up in that state (Dougherty, Hiserote, & Shaw, 2013, in press). Dougherty & Fleming (2012) 
examined the percentages of Far Off Track students who caught up in four multi-state student cohorts in grades 8-12 
and two statewide Arkansas cohorts (2005-2009 and 2006-2010) in grades 4-8. 
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Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Sawyer & Gibson, 2012). Efforts to close 
academic preparation gaps should begin as early as possible, be more intensive, and take as long 
as necessary. Even if starting earlier does not reduce the amount of time it takes for students to 
catch up, starting earlier gives them more time to catch up. 

In addition, a database might be developed to learn more about how effective various 
programs and interventions are at helping students catch up — from how far behind and over what 
length of time (Dougherty, 2010). Key components of the database would include information on 
how far behind the students are at different points in time and the nature, length, and intensity of 
the interventions they receive. From this, it might be possible to identify combinations of 
curriculum, interventions, and time requirements that are sufficient to enable most off-track 
students to succeed. 

These findings should also affect the requirements that accountability systems place on 
schools. For example, reasonable growth goals might be set based on student performance in 
more successful schools (ACT, 2009, 2012d), and goals for percentages of students reaching 
college and career readiness should take into account the students’ starting points and the number 
of years the school has available to catch them up, as is done in value-added models. In general, 
policy and practice should be infonned by data on the success of real students in actual schools. 
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Appendix A 


Narrowing of Scale Score Gaps in Grades 8-12 
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Narrowing of Scale Score Gaps in Grades 8-12 

Because the ACT Explore and ACT tests are scored on a common scale, growth between 
those two tests can be measured in scale score points. Accordingly, we disaggregated students in 
the longitudinal grades 8-11 and 8-12 cohorts into the scale score categories described in Table 
8. Figure A1 (on page 50) shows the percent of Far Off Track students in the four Arkansas 
cohorts falling into each of these categories. Figures A2, A3, and A4 (on pages 50-51) provide 
the same information for student demographic groups based on income, ethnicity, English 
language learner, and special education status. (The “Reached Benchmark” category in these 
charts shows the same statistics as in Figures 1-4.) 

As can be seen from these charts, the majority of Far Off Track students from every 
student group did not narrow their ACT Explore scale score gaps on the ACT. For example, 69% 
of low-income students did not narrow their scale score gaps in mathematics (Figure A2). The 
percentage of Far Off Track students in the first two scale score growth categories (reaching the 
Benchmark or closing their scale score gaps by half or more) was 9% in mathematics, 21% in 
reading, and 18% in science (Figure Al). For low-income students, the corresponding totals were 
7% in mathematics, 16% in reading, and 14% in science (Figure A2). These results are broadly 
similar to those from the z-score analysis. 34 All of this could indicate the presence of Matthew 
effects and the lack of sufficiently intensive and comprehensive interventions for these students 
in high school. 


33 Totals on the charts may differ from 100% and subtotals on the charts from those reported in the text, due to 
rounding. 

34 Apparent discrepancies between the results from the z-score analysis in Figures 5-8 and those from the scale score 
analysis in Figures A1-A4 are generally based on differences in how far behind the students started out in eighth 
grade. For example, Far Off Track special education students started out an average of 7.3 scale score points (2.1 
standard deviations) below the Benchmark in ACT Explore mathematics, versus 6.1 points (1.7 standard deviations) 
behind for all Far Off Track students. So even though a relatively high percentage of special education students 
(40%) narrowed their scale score gaps in mathematics between Explore and the ACT (Figure A4), most of those 
students did not narrow the gaps enough to get out of the Far Off Track group (Figure 8). 
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Figure Al. Percent of Arkansas Far Off Track students narrowing scale score gaps relative to the 
Benchmarks in grades 8-11 or 8-12, by subject. 
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Appendix B 


Results by Income, Ethnicity, and Gender 
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Table B1 

Percentage of All Arkansas ACT Explore Tested Students Who Were Far Off Track in 8 th 
Grade from 2006-07 and 2007-08 School Years 







Percentage of 8th graders who were Far 







Off Track 






Number of 




Group 

Income 

Ethnicity 

Gender 

students 

Mathematics 

Reading 

Science 

9 

Low- 

income 

African 

American 

Male 

Female 

3,440 

58.8% 

76.5% 

61.9% 

10 

3,752 

47.1% 

62.0% 

51.3% 

11 

Low- 

income 

Hispanic 

Male 

Female 

1,074 

41.4% 

59.9% 

43.1% 

12 


1,070 

37.9% 

48.9% 

37.9% 

13 

Low- 

Other 

Male 

5,230 

36.5% 

50.9% 

41.5% 

14 

income 


Female 

5,375 

29.4% 

39.7% 

33.8% 

15 

Non-low- 

income 

African 

American 

Male 

Female 

1,015 

49.3% 

65.5% 

51.9% 

16 

1,054 

34.6% 

47.6% 

37.8% 

17 

Non-low- 

income 

Hispanic 

Male 

Female 

244 

30.7% 

45.9% 

38.1% 

18 


224 

22.8% 

34.8% 

31.7% 

19 

Non-low- 

income 

Other 

Male 

8,482 

19.9% 

31.8% 

25.2% 

20 


Female 

8,392 

14.2% 

22.2% 

18.6% 
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Table B2 


Percentage of Arkansas Students in Longitudinal Cohorts Who Were Far Off Track in 8 th Grade 
from 2006-07 and 2007-08 School Years: Students Disaggregated by Income, Ethnicity, and 
Gender 


Percentage of 8th graders who were 
Far Off Track 


Number 


Group 

Income 

Ethnicity 

Gender 

of students 

Mathematics 

Reading 

Science 

9 

Low- 

African 

Male 

1,409 

41.2% 

65.6% 

46.5% 

10 

income 

American 

Female 

2,145 

35.0% 

52.5% 

42.1% 

11 

12 

Low- 

income 

Hispanic 

Male 

Female 

387 

485 

25.8% 

26.0% 

46.3% 

37.5% 

30.0% 

27.6% 

13 

Low- 

Other 

Male 

1,779 

16.9% 

34.3% 

22.8% 

14 



Female 

2,534 

16.7% 

26.6% 

21.7% 

15 

Non-low- 

African 

Male 

478 

35.6% 

53.8% 

42.3% 

16 

income 

American 

Female 

620 

27.7% 

39.5% 

30.0% 

17 

18 

Non-low- 

income 

Hispanic 

Male 

Female 

93 

120 

14.0% 

14.2% 

26.9% 

21.7% 

21.5% 

22.5% 

19 

Non-low- 

income 

Other 

Male 

5,088 

9.8% 

21.1% 

15.7% 

20 


Female 

5,909 

8.6% 

16.0% 

13.0% 


Percent of Arkansas Far Off Track 8 th Grade Students Meeting ACT College Readiness Benchmarks in Grades 11-12 
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Table BIO 


Percentage of All Tested Arkansas Students Who Were Far Off Track in 4" Grade from Students 
Taking Arkansas Benchmark Exams from2006-07 and 2007-08 School Years 


Group Income Ethnicity Gender 


Number of 
students 


Percent of 4th graders who 
were Far Off Track 

Mathematics Literacy 


9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 


Low- African 
income American 


Low- 

income 


Low- 

income 


Hispanic 


Other 


Non-low- African 
income American 


Non-low- 

income 


Non-low- 

income 


Hispanic 


Other 


Male 

Female 

Male 

Female 

Male 

Female 

Male 

Female 

Male 

Female 

Male 

Female 


6,634 

6,711 

2,534 

2,514 

11,451 

10,954 

1,081 

998 

435 

406 

13,241 

12,575 


63.4% 

73.4% 

58.5% 

60.6% 

47.9% 

65.5% 

46.7% 

53.9% 

38.4% 

53.5% 

37.0% 

39.5% 

47.6% 

53.3% 

40.4% 

38.0% 

38.2% 

52.9% 

40.8% 

41.6% 

20.1% 

31.2% 

18.7% 

20.2% 
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Table Bll 

Percentage of Arkansas Students in Longitudinal Cohorts Who Were Far Off Track in 4 th Grade from 
Students Taking Arkansas Benchmark Exams from 2006-07 and 2007-08 School Years 







Percent of 4th graders who 






were Far Off Track 





Number of 



Group 

Income 

Ethnicity 

Gender 

students 

Mathematics 

Literacy 

9 

Low- 

income 

African 

American 

Male 

Female 

4,553 

59.9% 

69.8% 

10 

4,948 

55.9% 

57.5% 

11 

Low- 

income 

Hispanic 

Male 

1,919 

43.6% 

62.0% 

12 


Female 

1,973 

44.0% 

51.5% 

13 

Low- 

Other 

Male 

8,325 

34.0% 

49.6% 

14 



Female 

8,483 

34.0% 

36.5% 

15 

Non-low- 

income 

African 

American 

Male 

Female 

797 

45.5% 

51.9% 

16 

805 

39.5% 

37.1% 

17 

Non-low- 

income 

Hispanic 

Male 

331 

33.8% 

48.3% 

18 


Female 

311 

38.3% 

39.0% 

19 

Non-low- 

income 

Other 

Male 

10,801 

18.3% 

29.2% 

20 


Female 

10,535 

17.3% 

18.8% 


66 


Table B 12 

Percentage of Arkansas Far off Track 4 th Grade Students Meeting College Readiness Benchmarks 
on 8 th Grade ACT Explore 






Mathematics 

Literacy/Reading 






Percent of 


Percent of 






Far Off 


Far Off 





Number of 

Track 

Number of 

Track 





Far Off 

students 

Far Off 

students 





Track 

reaching 

Track 

reaching 

Group 

Income 

Ethnicity 

Gender 

students 

Benchmark 

students 

Benchmark 

9 

Low- 

income 

African 

American 

Male 

Female 

2,727 

2.2% 

3,176 

1.8% 

10 

2,768 

1.7% 

2,843 

3.1% 

11 

Low- 

income 

Hispanic 

Male 

837 

4.8% 

1,183 

4.3% 

12 


Female 

869 

3.6% 

1,007 

5.9% 

13 

Low- 

income 

Other 

Male 

2,834 

4.1% 

4,125 

5.5% 

14 


Female 

2,886 

3.3% 

3,096 

6.9% 

15 

Non-low- 

income 

African 

American 

Male 

Female 

363 

1.7% 

414 

4.3% 

16 

318 

3.8% 

299 

4.3% 

17 

Non-low- 

income 

Hispanic 

Male 

112 

2.7% 

159 

8.8% 

18 


Female 

119 

2.5% 

120 

5.8% 

19 

Non-low- 

income 

Other 

Male 

1,973 

6.5% 

3,152 

9.0% 

20 


Female 

1,820 

6.2% 

1,985 

9.6% 


Arkansas Far Off Track 4 th Grade Students - Percentage in Mathematics z-Score Growth Categories between Grades 4 and 8 
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Arkansas Far Off Track 4 th Grade Students - Percentage in Literacy/Reading z-Score Growth Categories between Grades 4 and 8 
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